Videoconference communication device

ABSTRACT

A videoconference communication device prevents a screen from being unsightly to a viewer. A communication controller ( 103 ) is connected to respective partner devices ( 2 ), ( 3 ), and ( 4 ) through a network ( 5 ) and receives video data transmitted from respective partner devices ( 2 ), ( 3 ), and ( 4 ). A video and voice synthesizer ( 105 ) and a video and voice output controller ( 106 ) causes divided screens divided into a total number of sites in which a videoconference is performed, in display device ( 9 ) to be displayed before video data is received by the communication controller ( 103 ), and causes each region of the divided screens to sequentially display video data received by the communication controller ( 103 ).

TECHNICAL FIELD

The present disclosure relates to a videoconference communication devicewhich is provided at a host site and can be simultaneously connected topartner devices of a plurality of sites different from the host site.

BACKGROUND ART

In the related art, there is a known videoconference system capable ofbeing simultaneously connected to partner devices of a plurality ofsites so as to hold a videoconference (for example, PTL 1). Thevideoconference system decodes data transmitted from respective partnerdevices for each partner device, synthesizes video data each timedecoding is completed, and displays the synthesized video data.

CITATION LIST Patent Literature

PTL 1: Japanese Patent Unexamined Publication No. 2014-241525

SUMMARY OF THE INVENTION

A videoconference communication device according to the presentdisclosure adopts a configuration in which the videoconferencecommunication device that is provided at a host site and issimultaneously connectable to partner devices of a plurality of sitesdifferent from the host site includes a communication controller that isconnected to the respective partner devices through a network andreceives video data which is transmitted from the respective partnerdevices; and a display controller that displays divided screens dividedinto a total number of sites in which the videoconference is performedbefore the video data is received by the communication controller, andsequentially displays the video data which is received by thecommunication controller in each region of the divided screens.

According to the present disclosure, since synthesized video can bedisplayed from the start without changing a layout of a screen, it ispossible to prevent the screen from being unsightly to a viewer.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of avideoconference communication device according to Exemplary Embodiment1.

FIG. 2 is a block diagram illustrating a configuration of a video andvoice output controller according to Exemplary Embodiment 1.

FIG. 3 is a state transition diagram illustrating transition of a stateof a videoconference system according to Exemplary Embodiment 1.

FIG. 4 is a flowchart illustrating an operation of the videoconferencecommunication device according to Exemplary Embodiment 1.

FIGS. 5A to 5E are schematic views illustrating examples of videosdisplayed on a screen according to Exemplary Embodiment 1.

FIG. 6 is a block diagram illustrating a configuration of avideoconference system according to Exemplary Embodiment 2.

FIG. 7 is a flowchart illustrating an operation of the videoconferencecommunication device according to Exemplary Embodiment 2.

DESCRIPTION OF EMBODIMENTS

Prior to describing the embodiments, problems in the prior art will bebriefly described. A videoconference communication device in the relatedart displays synthesized videos while changing a layout of a screen eachtime decoding is completed, and thereby, an unsightly screen is producedfor the viewer.

An object of the present disclosure is to provide a videoconferencecommunication device that can prevent an unsightly screen from beingproduced for a viewer.

Hereinafter, embodiments of the present disclosure will be described indetail with reference to the drawings as appropriate.

(Exemplary Embodiment 1)

<Configuration and Connection Relationship of VideoconferenceCommunication Device>

First, a configuration and a connection relationship of videoconferencecommunication device 1 according to Exemplary Embodiment 1 will bedescribed in detail with reference to FIG. 1.

Videoconference communication device 1 and partner devices (conferenceparticipation sites) 2, 3, and 4 are installed at sites different fromeach other. Videoconference communication device 1 can be connected toand disconnected from partner devices 2, 3, and 4 through network 5. Thenumber of partner devices connected to videoconference communicationdevice 1 is not limited to three, and can be changed to two, four ormore as necessary.

Videoconference communication device 1 transmits/receives voice data andvideo data through network 5 after completing a connection to each ofpartner devices 2, 3, and 4. Network 5 is typically the Internet.

Videoconference communication device 1 is connected to user operationinput device 6, video input device 7, voice input device 8, and displaydevice 9.

User operation input device 6 detects an operation for selecting a siteperformed by a user, and transmits a signal including selected siteinformation indicating the selected site to videoconferencecommunication device 1 by wire or wireless manner. Typically, useroperation input device 6 is a remote controller or a touch panel.

Video input device 7 outputs video data and the like obtained bycapturing a site (hereinafter, referred to as a “host site”) wherevideoconference communication device 1 is installed to videoconferencecommunication device 1. The video data output from video input device 7includes video data stored in a PC, video data reproduced by a DVDplayer, and the like, in addition to the video data captured by acamera.

Voice input device 8 outputs voice data and the like obtained by pickingup voice of a host site to videoconference communication device 1. Thevoice data output from voice input device 8 includes voice data which isa set with the video data stored in the PC, voice data which is a setwith the video data reproduced by the DVD player, and the like, inaddition to the voice data picked up by a microphone.

Display device 9 displays the video data output from videoconferencecommunication device 1 on a screen, and outputs the voice data outputfrom videoconference communication device 1, from a speaker notillustrated.

Videoconference communication device 1 mainly includes user instructionreceiver 101, conference controller 102, communication controller 103,video and voice encoding/decoding unit 104, video and voice synthesizer105, video and voice output controller 106, still image holder 107,video input controller 108, and voice input controller 109. A displaycontroller is configured by video and voice synthesizer 105 and videoand voice output controller 106.

User instruction receiver 101 receives a signal transmitted from useroperation input device 6, extracts the selected site informationincluded in the received signal, and outputs the extracted informationto conference controller 102.

Conference controller 102 controls timing of data input to and outputfrom communication controller 103, video and voice encoding/decodingunit 104, and video and voice synthesizer 105, based on the selectedsite information input from user instruction receiver 101. In addition,conference controller 102 outputs the selected site information inputfrom user instruction receiver 101 to video and voice synthesizer 105.In addition, conference controller 102 controls signal transmissionprocessing and call establishment processing of communication controller103, and monitors whether or not video data is received, based on theselected site information.

Communication controller 103 operates at timing according to control ofconference controller 102. Communication controller 103 establishes acall with partner devices 2, 3, and 4. After establishing the call,communication controller 103 receives the video data and the voice datatransmitted from each of partner devices 2, 3, and 4 through network 5,and outputs the video data and the voice data to video and voiceencoding/decoding unit 104. In addition, after establishing the call,communication controller 103 transmits the video data and the voice datainput from video and voice encoding/decoding unit 104 to each of partnerdevices 2, 3, and 4 through network 5. Communication controller 103operates according to a predetermined communication protocol. Thecommunication protocol is typically SIP or H.323.

Video and voice encoding/decoding unit 104 operates at the timingaccording to the control of conference controller 102. Video and voiceencoding/decoding unit 104 encodes video data input from video and voicesynthesizer 105 and voice data input from voice input controller 109,and outputs the encoded data to communication controller 103. Inaddition, video and voice encoding/decoding unit 104 decodes the videodata and the voice data from partner devices 2, 3, and 4 which are inputfrom communication controller 103, and outputs the decoded data to videoand voice synthesizer 105.

Video and voice synthesizer 105 operates at the timing according to thecontrol of conference controller 102. Video and voice synthesizer 105generates synthesized video data for synthesizing and displaying videosin accordance with the number of sites of the selected site informationinput from conference controller 102, based on the video data input fromvideo and voice encoding/decoding unit 104, the video data input fromvideo input controller 108, and still image data read from still imageholder 107, and outputs the generated data to video and voice outputcontroller 106. In addition, video and voice synthesizer 105 generatessynthesized voice data by synthesizing the voice data from partnerdevices 2, 3, and 4 input from the video and voice encoding/decodingunit 104 and the voice data input from voice input controller 109, andoutputs the synthesized voice data to video and voice output controller106. In addition, video and voice synthesizer 105 outputs the video datainput from video input controller 108 to video and voiceencoding/decoding unit 104. Details of a configuration of video andvoice synthesizer 105 will be described below.

Video and voice output controller 106 causes the synthesized video datainput from video and voice synthesizer 105 to be displayed on a screenof display device 9 and causes a speaker of display device 9 to outputthe synthesized voice data as voice.

Still image holder 107 holds still image data for displaying apredetermined still image on the screen of display device 9 in advance.The predetermined still image is an image for notifying a currentcommunication situation with the partner device until the video data isreceived from each of partner devices 2, 3, and 4.

Video input controller 108 outputs the video data input from video inputdevice 7 to video and voice synthesizer unit 105.

Voice input controller 109 outputs the voice data input from voice inputdevice 8 to video and voice encoding/decoding unit 104 and video andvoice synthesizer 105.

<Configuration of Video and Voice Synthesizer>

Next, an internal configuration of video and voice synthesizer 105 willbe described with reference to FIG. 2. As illustrated in FIG. 2, videoand voice synthesizer 105 includes screen layout unit 201, duplicator202, and data allocator 203.

Screen layout unit 201 calculates a total value of the number of sites(partner devices) of the selected site information input from conferencecontroller 102, and outputs the total value to duplicator 202. Inaddition, screen layout unit 201 performs layout setting processing forsetting a screen layout for displaying divided screens divided into atotal number of sites in which a videoconference is performed and “1”(host site) is added to a total value of the number of sites, andoutputs screen layout information indicating the screen layout to dataallocator 203. The layout setting processing includes processing fordetermining a screen region for displaying the video data from eachpartner device.

At the start time, duplicator 202 duplicates the still image data readfrom still image holder 107 by the same number as the total value inputfrom screen layout unit 201, and outputs the duplicated data to dataallocator 203.

Data allocator 203 generates the synthesized video data by allocatingthe still image data input from duplicator 202 to the screen layout ofthe screen layout information input from screen layout unit 201, andoutputs the synthesized video data to video and voice output controller106. In addition, each time the video data from partner devices 2, 3,and 4 is input from video and voice encoding/decoding unit 104, dataallocator 203 generates newly synthesized video data by switching thestill image data into the video data. At this time, data allocator 203updates the synthesized video data without changing the screen layout.

<Transition of Connection State of Videoconference Communication Device>

Next, transition of a connection state of videoconference communicationdevice 1 will be described in detail with reference to FIG. 3.

Videoconference communication device 1 starts an operation by turning onpower.

Videoconference communication device 1 is in a non-communication stateshortly after the power is turned on (S1).

In a case of being connected to one partner device 2 in thenon-communication state (S1), videoconference communication device 1performs signal transmission to partner device 2 and performs signalacceptance from partner device 2, and thereby, a one-to-onecommunication state (1:1 communication state) with partner device 2 isset (S2). If the connection with partner device 2 in a communicationstate is cut in the 1:1 communication state (S2), videoconferencecommunication device 1 enters a non-communication state (S1).Videoconference communication device 1 ends an operation by turning thepower off in the non-communication state (S1).

In addition, in a case of being connected to still another partnerdevice 3 from the 1:1 communication state (S2), videoconferencecommunication device 1 performs signal transmission to partner device 3and performs signal acceptance from partner device 3, and thereby,entering a communication state (MCU communication state) with partnerdevice 2 and partner device 3 at a plurality of sites (S3).

If the connection with partner device 3 in the communication state iscut, videoconference communication device 1 enters a one-to-onecommunication state (S2), and furthermore, if the connection withpartner device 2 in the communication state is cut, videoconferencecommunication device 1 enters the non-communication state (S1).

In addition, videoconference communication device 1 performssimultaneous signal transmission to partner devices 2, 3, and 4 innon-communication state (S1), thereby, entering the communication state(MCU communication state) with partner devices 2, 3, and 4 (S3). If theconnections with partner devices 2, 3, and 4 are simultaneously cut,videoconference communication device 1 enters the non-communicationstate (S1).

A method of causing a user to manually input a signal transmissiondestination to videoconference communication device 1 at the time ofsignal transmission, a method of causing a user to select a list inwhich a plurality of pieces of signal transmission destinationinformation stored in videoconference communication device 1 in advanceare registered, or the like is considered as a method of performing thesimultaneous signal transmission. In addition, an IP address, atelephone number, an identification code, or the like is considered as amethod of specifying the signal transmission destination.

<Operation of Videoconference Communication Device>

Next, an operation of videoconference communication device 1 will bedescribed in detail with reference to FIGS. 4 and 5. FIG. 5 illustratesan example of a case where videoconference communication device 1performs the simultaneous signal transmission to partner devices 2, 3,and 4.

First, user instruction receiver 101 receives a signal from useroperation input device 6, extracts the selected site informationincluded in the received signal, and outputs the selected siteinformation to conference controller 102.

Conference controller 102 controls communication controller 103 suchthat the simultaneous signal transmission to partner devices 2, 3, and 4of N (N is an integer of 2 or more, N=3 in FIG. 5) sites of the selectedsite information is performed. Thereby, communication controller 103performs the simultaneous signal transmission to partner devices 2, 3,and 4 (S11). Specifically, in a case where the selected site informationis information of one list in which a plurality of pieces of signaltransmission destination information is recorded, conference controller102 can recognize site number N based on the number of signaltransmission destinations of the signal transmission destinationinformation registered in the list.

In addition, video and voice synthesizer 105 determines the screenlayout which is (N+1) divided screens, based on the selected siteinformation input from conference controller 102 (S12).

Next, video and voice synthesizer 105 reads the still image data fromstill image holder 107 and controls video and voice output controller106 so as to display the read still image data on each of the dividedscreens (S13). In a case where all the sites are before a response,video and voice synthesizer 105 displays the video of the host site ondivided screen B1 and displays a still image of “in course of signaltransmission” on the other divided screens B2, B3, and B4, asillustrated in FIG. 5A.

In a case where a condition that there is a site in which the video datais not received is satisfied, conference controller 102 controlscommunication controller 103 such that the signal transmission to thepartner device of the n-th (n is an integer equal to or less than N)site in which the video data is not received is performed. Thereby,communication controller 103 performs the signal transmission to thepartner device of the n-th site (S14).

Next, conference controller 102 determines whether or not a call isestablished in communication controller 103 (S15).

In a case where the call is not established in communication controller103 (S15: NO) as the power of the partner device of the n-th site isturned off or the like, conference controller 102 performs a control ofmaking video and voice synthesizer 105 display the still image. Thereby,video and voice synthesizer 105 reads the still image data from stillimage holder 107. Video and voice synthesizer 105 performs a control fordisplaying the still image of “in course of connection preparation” ondivided screen B2 of display device 9 (S16) as illustrated in FIG. 5B.Thereafter, the flow returns to the processing of S14.

Meanwhile, in a case where the call is established in communicationcontroller 103 (S15: YES), conference controller 102 performs a controlsuch that with respect to video and voice encoding/decoding unit 104starts encoding and decoding of the video data and the voice data (S17)and waits reception of the video data.

Next, conference controller 102 determines whether or not the video datais received from the partner device of the n-th site (S18).

In a case where the video data is not received from the partner deviceof the n-th site due to loss of a packet because of being in a lowbandwidth (S18: NO), conference controller 102 performs a control ofmaking video and voice synthesizer 105 display a still image. Thereby,video and voice synthesizer 105 reads the still image data from stillimage holder 107. Video and voice synthesizer 105 performs a control fordisplaying the still image of “in course of video preparation” ondivided screen B3 and B4 of display device 9 (S19) as illustrated inFIG. 5C. Thereafter, the flow returns to the processing of S14.

Meanwhile, in a case where the video data is received from the partnerdevice of the n-th site (S18: YES), conference controller 102 performs acontrol of making video and voice synthesizer 105 display the video datafrom the partner device of the n-th site (S20). Thereby, video and voicesynthesizer 105 performs a control of displaying the video on dividedscreens B3 and B4 of display device 9, as illustrated in FIG. 5D.

Conference controller 102, communication controller 103, video and voiceencoding/decoding unit 104, and video and voice synthesizer 105 repeatthe processing of S100 (S14 to S20) until the condition for performingthe processing of S100 is not satisfied.

Thereby, in a case where video data from all the sites is received,videos are displayed on all divided screens B1, B2, B3, and B4 asillustrated in FIG. 5E.

As described above, according to the present embodiment, it is possibleto display the synthesized video without changing a layout of a screenfrom the start, by displaying divided screens divided into the totalnumber of sites in which a videoconference is performed before videodata is received, and by sequentially displaying the received video dataon each of the divided screens. Thereby, it is possible to prevent thescreen from being unsightly to a viewer.

In addition, according to the present embodiment, it is possible for theviewer to recognize a current state until the video of each site isdisplayed, by displaying a still image notifying each of the dividedscreens of a current communication situation until video data from eachof the partner devices is received.

(Exemplary Embodiment 2)

<Configuration and Connection Relationship of VideoconferenceCommunication Device>

Next, a configuration of videoconference communication device 1 aaccording to Exemplary Embodiment 2 will be described in detail belowwith reference to FIG. 6. In videoconference communication device 1 aillustrated in FIG. 6, the same reference numerals or symbols will beattached to the same configuration parts as those of videoconferencecommunication device 1 illustrated in FIG. 1, and description thereofwill be omitted.

Videoconference communication device 1 a illustrated in FIG. 6 adopts aconfiguration in which video recording and voice recording controller301 is added to videoconference communication device 1 illustrated inFIG. 1. In addition, in videoconference communication device 1 aillustrated in FIG. 6, video and voice synthesizer 105 a is addedinstead of video and voice synthesizer 105 in FIG. 1.

In addition, videoconference communication device 1 a is connected tovideo recording and voice recording device 10 in video recording andvoice recording controller 301.

Video recording and voice recording device 10 records voice data andvideo data which are input from videoconference communication device 1 a(video recording and voice recording controller 301).

Video and voice synthesizer 105 a receives the video data from partnerdevices 2, 3, and 4 of all the sites, and outputs the synthesized videodata and the synthesized voice data to video recording and voicerecording controller 301 at timing when the synthesized video data isgenerated. The other functions of video and voice synthesizer 105 a arethe same as those of video and voice synthesizer 105.

Video recording and voice recording controller 301 records thesynthesized video data and the synthesized voice data which are inputfrom video and voice synthesizer 105 a in video recording and voicerecording device 10.

<Operation of Videoconference Communication Device>

Next, an operation of videoconference communication device 1 a will bedescribed in detail with reference to FIG. 7. In FIG. 7, the samereference numerals or symbols are attached to the steps common to thosein FIG. 2, and description thereof is omitted.

In FIG. 7, a configuration in which step S21 is added after step S100 isadopted as compared with FIG. 2.

At a timing when the processing up to S100 is completed and thesynthesized video data of the video data received from partner devices2, 3, and 4 of all the sites is generated, video recording and voicerecording controller 301 starts to record the synthesized video data andthe synthesized voice data which are input from video and voicesynthesizer 105 a in video recording and voice recording device 10(S21).

As described above, according to the present embodiment, since recordingof the synthesized video data and the synthesized voice data startsafter all the videos of a plurality of sites are displayed, it ispossible to prevent unnecessary video data and voice data from beingrecorded.

In the present embodiment, the present disclosure is not limited to acase where video data is displayed on each of divided screens of adisplay device each time the video data is received from a partnerdevice without changing a screen layout, and the above effects can beobtained even in a case where the video data is synthesized each timethe video data is received and the synthesized video data is displayedwhile the screen layout is changed each time the video data is received,as in Exemplary Embodiment 1.

In addition, in the present embodiment, recording of voice data maystart a little earlier than a timing when recording of the video datastarts, such as a timing when decoding of the video data of all thesites is completed, or a time when processing in layout settingprocessing ends up to a predetermined ratio (for example, 80%). Inaddition, in the present embodiment, the timing when recording of thevoice data starts may be able to be arbitrarily set by a user.

In the present disclosure, a type, an arrangement, a number, and thelike of members are not limited to the aforementioned embodiment, and itis natural that an appropriate modification, such as an appropriatereplacement of the configuration elements for obtaining the same actioneffects, can be made in a range without departing from the gist of thedisclosure.

Specifically, in Exemplary Embodiment 1 and Exemplary Embodiment 2, acase where a still image is displayed until video is displayed isdescribed, but the present disclosure is not limited to this, andcharacter information such as a message other than the still image maybe displayed.

In addition, in Exemplary Embodiment 1 and Exemplary Embodiment 2, acase where divided screens divided into the same size are displayed isdescribed, but the present disclosure is not limited to this and if thescreen layout is not changed, the divided screens divided into differentsizes or divided screens in which a part overlaps may be displayed. Inaddition, in the present disclosure, the divided screens may bedisplayed on a part of the screen, a region where the divided screensare not allocated may be a black image, and a still image may bedisplayed on a region where the divided screens in a screen of thedisplay device are not allocated.

In addition, in Exemplary Embodiment 1 and Exemplary Embodiment 2, thestill images are changed in the order of “in course of signaltransmission”, “in course of connection preparation”, and “in course ofvideo preparation”, but the present disclosure is not limited to this,and all the still images may be the same until the video is displayed.In addition, in the present disclosure, the still images other thanthose described above may be used as long as a content to be notified bythe still image also includes information that understands a currentcommunication situation.

INDUSTRIAL APPLICABILITY

The present disclosure is suitable for being used for a videoconferencecommunication device that is provided at a host site and can besimultaneously connected to partner devices of a plurality of sitesdifferent from the host site.

REFERENCE MARKS IN THE DRAWINGS

1, 1 a videoconference communication device

2, 3, 4 partner device (conference participation site)

6 user operation input device

7 video input device

8 voice input device

9 display device

10 video recording and voice recording device

101 user instruction receiver

102 conference controller

103 communication controller

104 video and voice encoding/decoding unit

105, 105 a video and voice synthesizer

106 video and voice output controller

107 still image holder

108 video input controller

109 voice input controller

201 screen layout unit

202 duplicator

203 data allocator

301 video recording and voice recording controller

The invention claimed is:
 1. A videoconference communication devicewhich is provided at a host site and is simultaneously connectable topartner devices respectively provided at a plurality of sites differentfrom the host site, the videoconference communication device comprising:a communication controller that is connected to each of the partnerdevices through a network and receives video data which is transmittedfrom each of the partner devices; and a display controller that causes ascreen layout of divided screens to be displayed before the video datais received by the communication controller, and causes each region ofthe divided screens to sequentially display the video data which isreceived by the communication controller in each region of the dividedscreens, wherein a number of the divided screens is a total number ofsites in which the videoconference is performed, the number of dividedscreens is determined before the video data is received by thecommunication controller, and the number of divided screens is fixedafter the number of divided screens is determined such that the numberof divided screens does not change from (i) when the number of dividedscreens is determined before the video data is received by thecommunication controller to (ii) when the videoconference is ended, andwherein before the communication controller receives video data from oneof the partner devices, the display controller displays, in a dividedscreen corresponding to the one of the partner devices, a still imageindicating a current communication situation with the one of the partnerdevices until the communication controllers receives the video data fromthe one of the partner devices.
 2. The videoconference communicationdevice of claim 1, wherein when there is an n-th site from among theplurality of sites from which video data is not received by thecommunication controller, the communication controller performs signaltransmission to the partner device at the n-th site, and thecommunication controller determines whether or not a call is establishedwith the partner device at the n-th site.
 3. The videoconferencecommunication device of claim 2, wherein when the communicationcontroller determines that the call is not established with the partnerdevice at the n-th site, the display controller displays, in the dividedscreen correspond to the partner device at the n-th site, a first stillimage indicating that indicating that the communication controller ispreparing to connect to the partner device at the n-th site until thecommunication controller determines that the call is established withthe partner device at the n-th site, and wherein when the communicationcontroller determines that the call is established with the partnerdevice at the n-th site, the display controller displays, in the dividedscreen correspond to the partner device at the n-th site, a second stillimage indicating that indicating that the communication controller ispreparing to receive the video data from the partner device at the n-thsite until the communication controller receives the video date from thepartner device at the n-th site.